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ABSTRACT 

We investigate whether a business curriculum develops critical thinking ability or at least serves 
as a filter for critical thinking (i.e., students who cannot think critically tend not to progress 
toward graduation). We measure critical thinking by performance on the Watson-Glaser Critical 
Thinking Appraisal Short Form which was administered to a sample of 600 students enrolled in a 
junior-level operations management course. We find that students who have completed more 
credit hours score significantly higher than those with fewer hours completed. This advantage 
appears specifically evident in the areas of interpreting information and evaluating arguments. 
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INTRODUCTION 


he ability to think critically has always been important for success in business. However, it is 
arguably becoming an increasingly vital skill in the modern economy. The speed and hyper 
competitiveness of business in the era of globalization imply, particularly for citizens of developed 
nations, that it is necessary to be able to observe, analyze, evaluate, and react creatively to changing situations. 



Business schools have long argued that they foster critical thinking skills, both through curriculum design 
and through course delivery. However, such claims are seldom backed by sound empirical design and 
experimentation. Moreover, even when rigorous methodology is applied, it can remain unclear whether students’ 
critical thinking skills are enhanced by the curriculum. One explanation is that progressing through the business 
curriculum enhances the critical thinking skills of students, which is the view implicit in most claims. However, it is 
also possible that the business curriculum merely filters for good critical thinkers. In that scenario, students who 
begin with poor critical thinking skills self-select into other majors, while only good critical thinkers progress 
toward graduation. However, the progress of good critical thinkers is based upon their a priori critical thinking 
skills; the business curriculum does little or nothing to enhance them. This paper seeks to distinguish between these 
competing explanations. 


LITERATURE REVIEW 


A very recent article in this journal, Behar-Horenstein and Lian Niu (2011), reviews the general literature 
regarding the teaching of critical thinking skills at the post-secondary level. Specifically, they argue that “future 
research should properly address internal validity threats, e.g. by adopting at least a quasi-experimental design, in 
order to establish causal relationship between intervention and changes in students’ critical thinking skills.” This 
paper is true to this notion through systematic quantitative analysis of students’ performance on a standardized test 
of several critical thinking measures across a diverse sample of students in a college of business. We also follow 
their recommendation by discussing the generality of our results without concluding that one size fits all. 

Rudinow and Barry (1994) define critical thinking as a process that emphasizes a rational basis for what we 
believe and provides standards and procedures for analyzing, testing and evaluating our beliefs. Williams (2002) 
provides a concise summary of the practice of successful critical thinking. He begins with the Rudinow and Barry 
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presumption that critical thinking is grounded in rationality then proceeds to identify the common barriers to 
successful critical thinking. These include, among others, the influence of our desires on the solutions we develop; 
various biases that influence our thinking; and the very difficult time that human beings have in accurately 
comprehending uncertainty and risk. He then provides a framework with which businesspeople and ordinary citizens 
can enhance their critical thinking skills. His recommendations include improving problem identification and idea 
generation, as well as examining proposed solutions for validity. He also focuses on the importance of creativity for 
problem-solving. The business curriculum, and particularly its quantitatively-oriented courses, is typically designed 
to teach students the same problem-solving behavior. Thus, we expect that it should lead to improved critical 
thinking by business students as they progress toward graduation. Other references have alternative approaches to 
teaching critical thinking skills. Stratton (1999) provides a more rigorous treatment of critical thinking for college 
students, but does it in a general context, rather than a business framework. Fisher (2001) provides readers with 
numerous examples designed to help them practice thinking critically as they work through the text. 

Giancarlo and Facione (2001) studies how attributes of critical thinking evolved in students attending a 
liberal arts university. Their data derive from the California Critical Thinking Disposition Inventory (CCTDI), 
which students took in their freshman year and again four years later. She observed significant differences in the 
scores for truth-seeking and critical thinking self-confidence, as well as in the overall CCTDI score. MacPherson 
(1999) offers pedagogy for using peer-graded oral presentations of literature reviews to assess and enhance critical 
thinking. She concludes that both the grading paradigm and the content of the literature review are influenced by 
critical thinking strategies and skills. 

While the literature on critical thinking in business is relatively small, the discipline of economics has a 
richer history of examining the link between learning economics and thinking critically. For example, Borg and 
Stranahan (2006) investigate the link between the economics knowledge gained in introductory economics courses 
(as measured by the well-known TUCE test) and students’ performance on the Watson-Glaser Critical Thinking 
Appraisal, the same exam that we use in our analysis. Their empirical results are consistent with the observation that 
students who acquire more economics knowledge during the economics principles course score higher on the critical 
thinking exam. Borg and Stranahan provide an excellent literature review, including some references linking 
technology-based learning exercises (e.g., on-line discussion groups) to critical thinking ability. The reader is 
referred there for additional information. 

Other disciplines are also investigating their influence on students’ ability to think critically. Among these, 
Miller (1992) employs pre- and post-testing of nursing students and finds a statistically significant positive 
incremental effect on critical thinking scores. Interestingly, they find that the nursing GPA was statistically 
significantly related to the post-test score, but that general education GPA was not. Her call for curriculum and 
teaching strategy innovations to enhance critical thinking skills is addressed, for example, by Niedringhaus (2001), 
who offers a holistic approach that assesses critical thinking skills through writing assignments for nursing students. 

Watson-Glaser Critical Thinking Appraisal 

The Watson-Glaser Critical Thinking Appraisal (or WGCTA), the initial version of which was developed 
in 1952 (Watson and Glaser, 1952) and the precursors of which were developed in 1925 and 1937 by Watson and 
Glaser, respectively (Watson and Glaser, 2006), is one of the most widely employed objective (i.e., multiple-choice) 
instruments for measuring critical thinking (Carrithers and Bean, 2008). Indeed, a Google Scholar search in May 
2008 revealed over 500 works in which the instrument has been cited and/or used since 1989. The WGCTA comes 
in two versions: a standard version containing 80 multiple-choice items that is designed to be completed in 60 
minutes (Pantesting.com, 2008), and a short form (first published in 1994) containing 40 multiple-choice items that 
is designed to be completed within 40 minutes, with 10 of these minutes for reading instructions and example 
problems (Watson and Glaser, 2006). Research on both instruments demonstrates that the short form yields results 
that are very similar to those on the standard form (Watson and Glaser, 2006). Because of its shorter completion 
time and lower cost, we administered the short form. 

Each version of the instrument is divided into sub-sections measuring five different aspects of critical 
thinking ability; making accurate inferences (referred to on the instrument as Test 1), recognizing assumptions (Test 
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2), making proper deductions (Test 3), interpreting information (Test 4), and evaluating arguments (Test 5) 
(Harcourt Assessment, 2008). The instrument is designed to measure critical thinking irrespective of content 
knowledge. It does not require mathematical computation and requires only a ninth-grade education (more than 98% 
of the words employed are at or below a ninth-grade reading level) (Watson and Glaser, 2006). 

METHODOLOGY 

During the spring, summer, and fall terms of calendar year 2007, the WGCTA short form was administered 
to 600 students enrolled in 19 sections of a required undergraduate core course in operations management at an 
urban comprehensive university with a total enrollment of approximately 17,000, and a business school 
undergraduate enrollment of approximately 3,000. These course sections included nine spring term sections, two 
summer term sections, and eight fall term sections. This course was selected for administration in part because of the 
cross section of students enrolled in it. As a required course for all business majors, it offered an opportunity to 
capture a near census of students with a comprehensive set of demographics. Also, because it is a junior-level course 
with no business prerequisites, some students take it early in their junior year - meaning for many it is one of the 
first business courses taken. However, for most business students it is not itself a prerequisite for later required 
courses, so many students delay taking the course until late in their program of study. The high level of quantitative 
applications and the perceived and actual rigor associated with the course material likely contributed to many 
students’ decisions to delay taking the course. Thus, the course presented an opportunity to capture a relatively 
broad cross-section of students in terms of majors, demographics, and credit hours taken. 

In all instances, the associate dean in charge of assessment and accreditation administered the exam, 
reducing any variation in outcomes that might be associated with differences in directions or presentation style. A 
75-minute class meeting was set aside to administer the exam and students were given approximately 40 minutes to 
complete the instrument (research on the WGCTA short form indicates that 90% of respondents complete the test 
within 30 minutes (Watson and Glaser, 2006)). All students attending during the selected class meeting were 
administered the paper-and-pencil version of the instrument. The associate dean preceded the administration with a 
presentation regarding the tests’ role and importance in the college’s accreditation processes, and as a quid-pro-quo 
for each student, promised an individualized report showing his/her scores as well as his/her percentile ranking vis- 
a-vis 11 different comparison groups from industry (including senior executives). Stated otherwise, taking the 
appraisal was presented as an opportunity for self-assessment for the student, and an opportunity to see how they 
currently compared to those already working in industry. Although students were occasionally given credit by the 
course instructor for simply taking the appraisal, in no case were students given any sort of course credit based on 
their actual performance on the appraisal. The lack of course credit approach likely reduced any motivation for 
cheating from fellow classmates or leakage of appraisal content from one course section to the next, although it is 
possible that some students did not take the appraisal seriously, with their scores suffering accordingly. 

On the appraisal, students were asked to provide their names and student identification numbers. These 
identification numbers were then used to match student results to demographic information on major, gender, and 
ethnicity, total credit hours completed at the institution at which the analysis was conducted, and total credit hours 
transferred into the institution from other schools. 

We conducted six separate analyses: one using the overall appraisal score as the dependent variable, 
operationalized as the number of items (out of 40) answered correctly, and the other five using the percentage of 
items answered correctly on each of the five tests (or sub-scores). The number of items within each test differed: 
there were seven items each within tests 1 and 4, eight within test 2, and nine each within tests 3 and 5. We are 
aware that Watson and Glaser indicate that it is the collective score that is a reliable critical thinking measure, and 
that the five sub-scores are not necessarily reliable as measures of specific critical thinking aspects (Watson and 
Glaser, 2006). However, we thought it of interest to investigate their relationship to our regressors nevertheless, as 
have previous researchers (e.g., Zascavage, et ah, 2007). 

The independent variable of interest for investigating our research question was the total number of credit 
hours completed, including credit hours completed at the target institution plus credit hours transferred from other 
institutions (because of the very large proportion of students in the study with some number of transfer hours. 
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narrowing the analysis to just non-transfers was not feasible.) Control factors included binary variables representing 
gender (with female as the omitted category), ethnicity (black, Hispanic, or Asian, with white as the omitted 
category), declared major (accounting, economics, finance, financial services, international business, marketing, and 
transportation/logistics, with the largest major of management as the omitted category), term administered (summer 
or fall, with spring as the omitted category), and instructor of the section in which the instrument was administered 
(there were four instructors, requiring three binary variables). Due to historical and anecdotal observation by the 
authors, we also hypothesized that transfer students - or those with larger proportions of transferred hours - may be 
less academically prepared than “native” students (or those with fewer transferred hours). We thus constructed an 
additional control variable representing the percentage of total credit hours that were completed at another 
institution. (Almost 89% of the students in our sample had at least some transfer credit hours, mostly from 
community colleges.) 

We also omitted 16 of the 600 observations in which students were reported as having less than 60 credit 
hours completed. Sixty hours represented half of the 120 credit hours needed for graduation, the hours needed to 
rise to junior-level status, the number of lower-division (or non-business) credit hours required of all students, and 
the number of hours needed by community college students to qualify for automatic transfer into the institution at 
which the study was conducted. This approach tended to assure that each student included in the analysis had a 
baseline (and largely non-business) level of college coursework, and left us with 584 observations for analysis. 

Summary descriptive statistics for all variables appear in Table 1. Our sample contained almost 53% males, 
and approximately 29% were minorities. The average percentage of total credit hours that were completed at other 
institutions was 52.4%. The 584 students included in the study averaged 27.67 out of 40 items answered correctly, 
or 69.2%. Two students scored a perfect 40 of 40. 

We used ordinary least squares (OLS) regression, and SAS Version 9.1, to estimate parameter coefficients 
for each of the six models. Although the dependent variable was limited to values between 0 and 40 for the model 
of the overall score, and to values between 0 and 100 for the models of the five sub-scores, limited dependent 
variable techniques yielded quite similar results to those reported for OLS. We employed a significance level of 
0.05 on all hypothesis tests. When testing the coefficient of total credit hours completed, we hypothesized a positive 
relationship between that factor and the number of WGCTA items (or the percentage of items on each test) 
answered correctly. 

RESULTS 

All OLS coefficient estimates appear in Table 2. The coefficient of total credit hours completed was 
positive and significant at the 0.05 level. This result supports our hypothesis that a business curriculum does 
develop critical thinking ability, or at least serves as filter for critical thinking ability. However, examination of the 
coefficient for credit hours completed in each of the five sub-score models reveals some potential insights into the 
specific areas of critical thinking that appear to be developed or filtered in coursework. (We reiterate Watson and 
Glaser’s cautions regarding the reliability of the sub-scores as specific measures, however.) Whereas credit hours 
completed are not significantly related to the ability to make accurate inferences (test 1), recognize assumptions (test 
2), or make proper deductions (test 3) - although coefficients do take the expected positive sign in each case - 
coursework completed is significantly related to interpreting information (test 4) and evaluating arguments (test 5). 
This is particularly true of interpreting information, which had by far and away the largest coefficient value and the 
highest significance level. This result is made more interesting by noting from Table 1 that average student scores 
were the second lowest in this area (at 65.4%), among all five sub-scores. 

In terms of our control variables, neither declared major, course instructor, nor the term in which the 
appraisal was administered were significantly related to WGCTA performance overall or in any of the sub-scores, 
save one instance: Financial Services majors differed on making accurate inferences. 

However, in contrast to the reports of Watson and Glaser (2006), our results exhibit a bias in favor of 
males. However, review of the coefficients for males in each sub-score model indicates that this bias appears 
significant only for interpreting information, where the coefficient value indicates a relatively large 11.35 percentage 
point advantage for males versus females. 
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Table 1: Summary Statistics for All Variables (n=584) 


Variable 

Mean 

Standard Deviation 

Minimum 

Maximum 

WGCTA items correct 

27.6695 

5.5021 

13 

40 

Percent correct on Test 1 

58.1703 

24.7010 

0 

100 

Percent correct on Test 2 

70.7406 

27.7883 

0 

100 

Percent correct on Test 3 

73.9916 

19.6597 

0 

100 

Percent correct on Test 4 

65.3620 

21.7282 

0 

100 

Percent correct on Test 5 

74.4863 

17.6539 

0 

100 

Total credit hours 

91.4005 

20.2708 

60 

254 

Male 

0.5274 

0.4997 

0 

1 

Black 

0.1233 

0.3290 

0 

1 

Hispanic 

0.0908 

0.2875 

0 

1 

Asian 

0.0702 

0.2557 

0 

1 

White 

0.7089 

0.4547 

0 

1 

Management major 

0.3630 

0.4813 

0 

1 

Marketing major 

0.1079 

0.3105 

0 

1 

Accounting major 

0.1695 

0.3755 

0 

1 

Finance major 

0.2021 

0.4019 

0 

1 

International business major 

0.0788 

0.2696 

0 

1 

Trans/Logistics major 

0.0462 

0.2102 

0 

1 

Financial services major 

0.0205 

0.1420 

0 

1 

Economics major 

0.0120 

0.1089 

0 

1 

Instructor 1 

0.1130 

0.3169 

0 

1 

Instructor 2 

0.4538 

0.4983 

0 

1 

Instructor 3 

0.0531 

0.2244 

0 

1 

Instructor 4 

0.3801 

0.4858 

0 

1 

Spring 

0.5428 

0.4986 

0 

1 

Summer 

0.1010 

0.3016 

0 

1 

Fall 

0.3562 

0.4793 

0 

1 

Proportion hours transferred 

0.5235 

0.3325 

0 

1 


Table 2: Ordinary Least Squares Coefficients (n=584) 

Reported significance levels are two-tailed, except those for total credit hours completed, which are one-tailed in the hypothesized (positive) 
direction. Test 1 = accurate inferences, test 2 = recognizing assumptions, test 3 = making proper deductions, test 4 = interpreting information, 
and test 5 = evaluating arguments. 

Variable 

Overall 


Test 1 


Test 2 

Test 3 


Test 4 


Test 5 

Intercept 

26.0642 

*** 

59.2410 

*** 

66.2660 *** 

76.8090 

*** 

50.1330 

*** 

68.8200 *** 

Total credit hours completed 

0.0259 

* 

0.0356 


0.0667 

0.0282 


0.1340 

** 

0.0690 * 

Male 

1.1274 

* 

-0.1310 


2.3480 

1.6830 


11.3450 

*** 

0.0359 

Black 

-3.3355 

*** 

-8.8240 

** 

-9.2090 * 

-10.2700 

*** 

-9.8060 

*** 

-4.1160 

Hispanic 

-2.2233 

** 

-4.3490 


-7.5710 

-6.4160 

* 

-5.4660 


-3.9240 

Asian 

-2.6620 

** 

-8.4910 

* 

-4.9050 

-10.5330 

** 

-6.9440 

* 

-2.6800 

Marketing major 

-0.5697 


-2.7540 


-3.6400 

-0.6920 


-0.1640 


-0.1340 

Accounting major 

0.3453 


0.0885 


2.8710 

-0.4450 


2.4610 


-0.2540 

Finance major 

0.3739 


0.4940 


-1.6650 

2.8790 


3.0160 


0.0265 

International business major 

-0.0014 


1.5250 


-2.5830 

-1.8660 


2.5880 


0.9470 

Trans/Logistics major 

1.2550 


1.6590 


2.6440 

1.0350 


5.1260 


5.2810 

Financial services major 

2.0841 


15.8240 

* 

10.4860 

1.6300 


7.3280 


-5.8020 

Economics major 

-0.0237 


-2.7790 


4.2780 

8.3680 


-5.4100 


-6.0640 

Instructor 1 

0.9684 


-0.6080 


0.3190 

2.1050 


5.4640 


4.5950 

Instructor 2 

0.4276 


3.0090 


0.9940 

-0.4080 


2.5170 


-0.0227 

Instructor 3 

0.1695 


0.0420 


2.9320 

1.8540 


-0.6110 


-2.1350 

Summer 

-0.9898 


-4.5840 


-3.5310 

-2.1220 


-4.8470 


1.5990 

Fall 

-0.5477 


-3.7850 


-1.2130 

-2.4170 


-1.1870 


1.2770 

Proportion of transfer hours 

-1.4337 

* 

-3.7610 


-0.9320 

-6.1920 

* 

-5.9950 

* 

-1.3230 


*** Significant at 0.001 level 
** Significant at 0.01 level 
* Significant at 0.05 level 


Again in contrast to Watson and Glaser (2006), we also find a minority bias, as the coefficients of all three 
minority categories were significant and negative in the model of overall scores. This bias was significant across 
four of the five sub-scores for black students and across three of the four sub-scores for Asian students. The scores 
for Hispanic students differed significantly only on making proper deductions. The only area in which no significant 
differences were found between white and minority students was in evaluating arguments. 
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Finally, we find that students with higher proportions of transfer hours do indeed perform significantly 
worse overall than those who transfer in fewer hours. Stated more generally, non-transfers (or those who transfer 
fewer hours) tend to do better than transfers. More specifically, this difference is significant when it comes to 
making proper deductions and interpreting information (in addition, the coefficients for this factor in the other sub- 
score models are consistently negative, although not significantly so). As a large number of the transfer hours come 
from the community college system, this result may very well be a proxy for general academic preparation and/or 
specific ability in the first place. However, it also raises questions as to how well these schools develop or filter 
critical thinking ability, particularly in those two areas. 

CONCLUSIONS 

Our findings show a significant relationship between credit hours completed and critical thinking ability. 
However, our analysis involved a convenience sample and did not involve a true pre-test/post-test equivalent group 
experimental design in which the same subjects were analyzed early and then again late in their program of study. 1 
Because of this, determining whether the curriculum developed this ability versus whether it simply filtered out 
those students with lesser abilities is a matter of some conjecture. 2 However, recent reports from the dean of the 
college in which this study was conducted indicate that the retention of business majors is quite high once they reach 
their junior level coursework (i.e., the level at which we administered the instrument). If retention is indeed high 
from that point, it would imply that our sub-sample of students with high (or higher credit hours) had not been 
significantly filtered. That would also tend to imply that the curriculum is developing critical thinking ability and 
not simply serving as a hurdle. 
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